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Abstract 



We have searched for the rare decays rj e~^e~r] , rj' e~^e~Tr^, rj' — > e~^e~^, 
and rj' — > e/x in hadronic events at the CLEO II detector. The search is 
conducted on 4.80 fb^^ of e+e^ colhsions at 10.6 GeV center-of-mass energy 
at the Cornell Electron Storage Ring (CESR). We find no signals in any of 
these modes, and set 90% confidence level upper limits on their branching 
fractions: B{r]' e^e'Tj) < 2.4 x lO'^, B{r]' e+e"7r°) < 1.4 x lO'^, 
B{r]' e+e-7) < 0.9 X 10~3, and B{r]' e^i) < 4.7 x 10"^. We also 
investigated the Dalitz plot of the common decay mode r/' — > vr^vr^r/. We fit 
the matrix element with the Particle Data Group parameterization |Mp=A| 
1 + ay p, where y is a linear function of the kinetic energy of the rj, and find 
Re(a) = -0.021 ± 0.025. 
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The 7] and t]' mesons share the same quantum numbers, and can both be used to inves- 
tigate C and CP violation, leptoquarks, lepton family violation, chiral perturbation theory, 
and other topics But while the t] has been the subject of several experiments, our ex- 
perimental knowledge of the t]' is limited; many measured upper limits for its decays are 
at the percent level 0. In this Letter we present searches for the rare decays rj' e~^e~ri, 
1]' — > e+e^TT*^, 7]' — > e"'"e~7, and rj' e/i produced in e~^e~ collisions at 10.6 GeV center- 
of-mass energy. We have concentrated on decays involving at least one electron because 
combinatoric backgrounds in hadronic events are much larger for photons and charged pions 
than for leptons. 

The decays rj' —>■ e~^e~ri and r/' e^e^n^ can occur with one or two intermediate virtual 
photons, as shown in Figure |I[ The rj', rj, and 7r° are even eigenstates of C, while a photon 
is C-odd; thus the one-photon process will be C-violating and the two-photon process C- 
conserving. Because B{ri' — > 71*^77) < 8 x 10~^, and the rates for these decays should be 
orders of magnitude smaller any signal at the 10~^ level or larger would signify a large 
C-violating contribution or other new physics. The current 90% confidence upper limits for 
these decays are 1.1% for e^e'r] and 1.3% for e+e~7r°. 

The Dalitz decays of the 77 and 7r° have both been measured, with branching fractions two 
orders of magnitude smaller than the 77 decays of the two mesons. Because B{ri' —>■ 77) = 
(2.12 ± 0.13)% 0, we may expect the rj' Dalitz decay at the 10~^ level. The decay rj' e/x 
with no accompanying neutrinos is an example of lepton number violation. The theoretical 
upper bound for this decay is on the order of 10~^^, determined from experiments on /i~ — > e~ 
conversion in heavy nuclei ||^ . A host of intermediate particles, including massive neutrinos 
or leptoquarks 0, could manifest themselves through a larger branching fraction for this 
decay. No measured upper limit has been published for either 77' — >■ e^e~7 or r]' — > e/i. 

The common decay r]' tt^-k^i] [i] 77) was used both as a normalization for our rare 
decay searches and to investigate the structure of the 77'. Defining the Dalitz variables 

1/= [2 + K/m^)]^-l, x = ^(T^, -T^J (1) 

in which T represents kinetic energy in the rj' rest frame and Q = rrirji — {mj^ + 2m.^), we fit the 
matrix element with the Particle Data Group parameterization, |Mp= A{\ 1+ay p +cx'^). 
Of particular interest is the real component of the complex constant a, which is a linear 
function of the kinetic energy of the 7]. A non-zero value of a may represent the contribution 
of a gluon component to the rj' decay [^]. There are two published measurements of Re(Q;); 
-0.08 ± 0.03 for r]' ^ r]n+n' 0] and -0.058 ± 0.013 for t]' r/7r°7r° §. Though these 
measurements are consistent, a need not be the same for the two decays. 

Data were collected using the CLEO II detector |^ at the Cornell Electron Storage Ring 
(CESR). We used 3.11 fb-^ at the T(45) resonance, 10.58 GeV, and 1.69 fb'^ at 10.52 GeV. 
Approximately (15±3)% of our rj' sample comes from BB decays. Charged particle momenta 
are measured in a 67-layer tracking system immersed in a 1.5 T solenoidal magnetic field. 
The main drift chamber also determines a track's specific ionization (dE/dx), which aids in 
particle identification. A 7800-crystal Csl calorimeter detects photons and is the primary 
tool for electron identification. Muons are identified using proportional counters placed at 
various depths in the steel return yoke of the magnet. 
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Selection criteria for rj' —>■ 7t~^7[~i] {i] 77) were as follows. We restricted our search to 
hadronic events by requiring at least five charged tracks in the drift chamber. Photons from 77 
candidate decays were required to be in the region of best energy resolution in the calorimeter 
(80% of the solid angle), to have an energy of at least 200 MeV, and not to overlap any noisy 
crystals or the projections of charged tracks. We rejected both photons of pairs with an 
invariant mass within 12.5 MeV/c^ of the vr*^ mass. Candidate pions must be well-tracked and 
must originate in the interaction region, within ±3mm radially in the plane perpendicular to 
the beam and within ib2cm in the beam direction. Because we intend to use these decays as 
normalization of our search for decays containing electrons, some requirements are designed 
to reduce systematic uncertainties in electron identification. We veto tracks which have a 
vertex with another track in the beam pipe or tracking chamber walls, because we will need 
to veto gamma conversions when seeking rj' decays involving electrons. Similarly, tracks 
projecting to the calorimeter endcap are rejected, because electron identification degrades in 
this region. 

We kinematically fit each photon pair with invariant mass within 40 MeV/c^ of the 1] mass. 
By requiring the t] and t]' to have minimum momenta of 0.6 and 1.0 GeV/c, respectively, we 
limit random combinations of real r/'s and charged pions, which are our largest background. 
The final signal of roughly 6700 events is shown in Figure From a Monte Carlo sample of 
1.3 X 10^ events we calculate an efficiency of (2.96 ±0.08)%, indicating that about 1.3 x 10^ 
rj' mesons were produced in our sample. 

The searches for rj' —>■ e^e~r] [rj 77) and rj' e+e~7r° (vr^ 77) use the same criteria 
as the normalizing mode, except that we now require the charged tracks to be positively 
identified as electrons. Our electron identification algorithm utilizes shower energy, track 
momentum, specific ionization loss, and shower shape to achieve an efficiency of roughly 
90% with a fake rate from charged pions of less than 0.5%. Signal efficiency is estimated 
from Monte Carlo samples for each decay, fitting each with a Gaussian and defining the 
signal region to be 2a to either side of the mean. Average background is determined by 
interpolating the data between 0.85 and 1.05 GeV/c^ in invariant mass, excluding the signal 
region. 

The data (solid) and Monte Carlo (dashed) results for e~^e~7] and e^e^n^ are shown in 
Figures |^a and ^d, respectively. Neither shows any statistically significant signal in data. 
The efficiency for the e~^e~ri mode is estimated from Monte Carlo to be (0.27±0.02)%. There 
is one event in the signal region with an expected background of 3.0 events; Poisson statistics 
gives us a 90% confidence upper limit of 2.8 events. The efficiency for the e"'"e~'7r° mode is 
(0.37 ± 0.02)%. There are 23 signal-region events and an average background of 31.0 events 
for an upper limit of 6.2 events at 90% confidence level. The background for this mode is 
larger due to the greater number of neutral pions available for random combinations. 

The Monte Carlo samples used above distribute the decay products evenly over phase 
space. One might argue that this is not appropriate for the one-photon C- violating diagrams 
for these decays, and that we may be unknowingly cutting away a signal of new physics. To 
check this, we generated a second Monte Carlo sample for each decay, using the measured 
electron-pair mass distribution from the Dalitz decay of the t] 0. These Monte Carlo samples 
give us estimated efficiencies of (0.21 ± 0.02)% for e^e~ri and (0.31 ± 0.03)% for e~^e~7T^. 
For each mode we will use the mean of the two Monte Carlo estimates as the efficiency and 
introduce half the difference as a systematic uncertainty. 
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For the rj' —>■ e~^e~'j search we require a minimum photon energy of 0.6 GeV to re- 
duce combinatoric background. We also see a significant background from random photons 
combined with an electron pair from the Dalitz decays of a vr*^ or 77. We reduce this back- 
ground by vetoing any event with an e"'"e~7 combination with invariant mass within 3a (21 
MeV/c^) of the vr*^ mass or within 2a (26 MeV/c^) of the 77 mass. The data (solid) and 
Monte Carlo results are shown in Figure the Monte Carlo sample again uses the electron- 
pair mass distribution from the 77 Dalitz decay. The efficiency for this mode is estimated to 
be (1.01 ± 0.06)%. There is again no significant signal, with 51 signal-region events and a 
background of 53.6 events, for a 90% confidence upper limit of 11.7 events. 

The criteria for muon identification in t]' — > efi are identical to those for pions, with the 
additional requirement that the muon penetrate three interaction lengths of material outside 
the calorimeter (roughly a 1.0 GeV/c momentum requirement). The results of this search are 
shown in Figure |^. The efficiency is found to be (4.92 ± 0.15)%. There are 650 events with 
an average background of 672 events, for an upper limit of 30.1 events at 90% confidence. 

Table | shows all sources of systematic uncertainty in our analyses which do not cancel 
in the normalization to the rj' tt+tt^t] mode. We used Poisson statistics to generate a 
confidence level distribution for the number of signal events in each mode. We then ran a 
simulation which assigned an error to each probability in the distribution, with the errors 
corresponding to a Gaussian with a width determined by the total uncertainty for that 
mode. This 'smearing' of the distribution increased the number of signal events at the 90% 
confidence level to the amount shown in Table ^ for each mode. Normalizing the signal limit 
and efficiency for each mode to that of r/' -k^ti't] allows us to calculate 90% confidence 
upper limits on the rare decay branching fractions: 

Biji' e+e-f]) < 2.4 x 10-^ (2) 
B{r]' e+e"7r°) < 1.4 x 10'^ (3) 
B{ri' e+e-7) < 0.9 X 10"^ (4) 

B{ri' e/i) < 4.7 x 10"^ (5) 

In measuring Re(a) we use the selection criteria for 77' tt^tt^i] [i] 77) with the 
gamma conversion and calorimeter endcap requirements removed for greater efficiency. We 
fit the T/vr+vr" signal in data with a Gaussian and define the signal region as 2a to either 
side of the mean. Background is removed by sideband subtraction; we use three different 
definitions of sideband (3 — 5a, 4 — Qa, and 5 — 7cr from the mean). Our sensitivity to 
the coefficient of x'^ is small, so we plot the projection along the y-axis for each choice of 
sideband. Because the requirement on the rj momentum may be biasing the distribution of 
T^, we also measure Re(a) with minimum rj momenta of 0.4 GeV/ c and 0.8 GeV/ c. Efficiency 
corrections are made by dividing our results by those from a Monte Carlo sample of 9.3 x 10^ 
rj' —>■ TT^TT~1] [t] —>■ 77) decays. We fit the results with a line whose slope will be twice 
the real component of a. One such plot is shown in Figure |[ The slope varies by ±0.012 
with choice of sideband, ±0.006 with choice of 7/ momentum requirement, ±0.010 from the 
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statistics of the Monte Carlo sample, and ±0.018 due to the statistical uncertainty of the fit. 
Adding these uncertainties in quadrature yields a result of Re (a) = —0.021 ± 0.025. While 
consistent with previous measurements, this result is much smaller and is also consistent 
with zero. 

In summary, we have searched for rare decays of the rj' involving at least one electron 
in e~^e~ collisions at the CLEO II detector. We find no statistically significant signals, 
and have assigned 90% confidence upper limits of B{ri' e~^e~ri) < 2.4 x 10^'^, B{ri' 
e+e-7r°) < 1.4 x lO^'l B{r]' e+e-7) < 0.9 x lO^'l and B{ri' e^) < 4.7 x 10"^ These 
measurements represent an order of magnitude improvement over previous limits, but are 
at least another order of magnitude short of discerning the Dalitz decay or suggesting a 
difficulty for the Standard Model. Wc have also examined the structure of the rj' 7r~^7r~r) 
and measured Re(a) = —0.021 ± 0.025, a value consistent with but much smaller than 
previous measurements. 

We gratefully acknowledge the effort of the CESR staff in providing us with excellent 
luminosity and running conditions. This work was supported by the National Science Foun- 
dation, the U.S. Department of Energy, the Research Corporation, the Natural Sciences and 
Engineering Research Council of Canada, the A. P. Sloan Foundation, the Swiss National 
Science Foundation, and the Alexander von Humboldt Stiftung. 
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5.9% 
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TABLE I. Summary of systematic uncertainties which do not cancel in normalization. 



Rare 


Limit without 


Limit with 


decay 


systematics (events) 


systematics (events) 


rj' e^e~r] 


2.8 


3.0 


rj' — e'^e~Tr^ 


6.2 


6.4 


rj — > e'^e~7 


11.7 


11.9 


rj — ^ e/Lt 


30.1 


30.4 



TABLE II. Results of applying systematic uncertainty to confidence level distribution. 
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FIG. 2. Measured invariant mass distribution from CLEO II data for rj' vr+vr r] (rj ^ 77). 
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FIG. 3. Invariant mass distributions for r/' decays to (a) e~^e~r], (b) e'^e~7T^, (c) e^e~7, and 
(d) e/i. The solid histogram represents the measured results, while the dashed curve is the Monte 
Carlo prediction with arbitrary normalization. 
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